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Abstract 

The aim of this paper is to introduce the construction of codes for a general discrete stationary memoryless 
04 , multiple access channel based on the the notion of the hash property. Since an ensemble of sparse matrices has a 

hash property, we can use sparse matrices for code construction. Our approach has a potential advantage compared 
CNj ■ to the conventional random coding because it is expected that we can use some approximation algorithms by 

^ using the sparse structure of codes. 

O: 

. Index Terms 

(N 

Shannon theory, hash property, linear codes, LDPC codes, sparse matrix, minimum-divergence encoding/decoding, 
multiple access channel. 

I. Introduction 

This paper describes the construction of multiple access channel codes. In a multiple access channel, two 



C/3 



X 



■ or more senders send messages to a common receiver The capacity region has been derived in [1][15] for a 
scenario where two senders have different private messages but no common message to be sent. This work has 

. been extended in [26] to a scenario where two senders have different private messages and a common message to 

■ be sent. The capacity region for two or more senders has been described in [6, Section 15.3.5][10, Chapter 4] in 

■ which there is no common message. In [12], the capacity region has been derived for a general multiple access 
channel in which two or more senders have messages common to some users. Applications of Low Density 
Parity Check (LDPC) codes to a multiple access channel have been introduced in [3][16][17]. Furthermore, 
there are many theoretical/experimental studies regarding the construction of multiple access channel codes by 
using LDPC codes, e.g. [2] [25]. It should be noted that they assumed channel noises to be additive. 

A contribution of this paper is to construct codes based on the notion of the hash property [22] [21], which 
is a stronger version of that introduced in [19] [20]. Another contribution is to construct codes by using sparse 
matrices for a general discrete memoryless multiple access channel including asymmetric one. We construct 
codes for the following scenarios: 

• Two or more senders have different private messages (Fig.l) [10, Theorem 5 in Chapter 4][6, Section 
15.3.5], 
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Fig. 1. Multiple Access Channel Coding: Private Messages 




Fig. 2. Multiple Access Channel Coding: Multiple Common Messages 



• Two or more senders have messages common to senders (Fig. 2) [12], and 

• Two senders have different private messages and a common message (Fig. 3) [26], 

where additive channel noises are not assumed. It should be noted that the first scenario includes two-sender 
scenario studied in [1][15]. The last scenario is included in the second scenario but we will discuss it separately 
because their code constructions are different. The proof of all the theorems is based on the notion of the hash 
property. It is an extension of the ensemble of the random bin coding |5|, the ensembles of linear matrices [7], the 
universal class of hash functions [9], and the ensemble of sparse matrices [18]. We use two lemmas, one related 
to 'saturation property'' (if the number of items is greater than the number of bins then there is an assignment 
such that every bin contains at least one item) and the other related to 'collision-resistance property'' (if the 
number of bins is greater than the number of items then there is an assignment such that every bin contains at 
most one item) proved in [19][21], where the lemma related to the 'collision-resistance property' is extended 
from a single domain to multiple domains. They are reviewed in Section IV. The saturation property is used to 
analyze the average encoding error and the extended collision-resistance property is used to analyze the average 
decoding error. It should be noted that the functions need not be linear for the hash property but it is expected 
that the space and time complexity of codes can be reduced compared with conventional constructions by 

Un [19], they were called 'saturating property' and 'coUision-resistant property,' respectively. We changed these terms following the 
suggestion of Prof T.S. Han. 
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Fig. 3. Two-user Multiple Access Channel Coding: Private and Common Messages 

assuming the linearity of functions. In fact, we can construct codes by using sparse matrices (with logarithmic 
column degree) because an ensemble of sparse matrices has a hash property [21]. Although the implementation 
of encoding and decoding operations of our approach is still intractable, our approach has a potential advantage 
compared to the conventional random coding presented in [6] [7] [10] [12] [26] because it is expected^ that we 
can use some approximation algorithms such as the sum-product algorithm [14] and the linear programing 
algorithm [11] for encoding and decoding operations in the proposed code with sparse matrix. 

II. Definitions and Notations 

Throughout this paper, we use the following definitions and notations. The cardinaUty of a set U is denoted 
by denotes the complement of U, and U\V = U f\V'' denotes the set difference. 

Column vectors and sequences are denoted in boldface. Let Au denote a value taken by a function A : — )• 
U at u gU", where is the domain of the function and U is the range of function. It should be noted that 
A may be nonUnear. When A is a linear function expressed by an / x n matrix, we assume that U = GF(g') 
is a finite field and the range of functions is UK For a set A of functions, let ImA be defined as 



We define a set CA{a) as 



lmA= U {Au : u&U"}. 
AeA 



Cyi(a) = {u : Au = a} 



In the context of linear codes, C/i(a) is called a coset determined by a. The random variables of a function A 
and a vector a € Im^ are denoted by the sans serif letters A and a, respectively. It should be noted that some 
random variables are denoted by the Roman letter (e.g. M, U, V, X, Y) which does not represent a function. 

For random variables U and V, let be the probability distribution and Huiv be the conditional probability 
distribution. Then the entropy H{U), the conditional entropy H{U\V), and the mutual information I{U;V) 
are defined as 

H{U) = Y,l^u{u)\og^— 
H{U\V) = ^ iJ,u\v{u\v)lJ.v{v) log 



IJ,u\v{u\v) 



Hn fact, the direct application of [11][14] does not provide good performance. Implementation of our approach is left for a future 
challenge. 
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I{U;V) = H{U) - H{U\V) 

where we assume a logarithm with base 2 when the subscript of log is omitted. For random variables U, V, 
and W, let /([/; y|VF) be the conditional mutual information defined as 

I{U; V\W) = H{U\W) - H{U\V, W). 

For u gU" and v e V", let Vu and Uu\v be the empirical distributions defined as 

^ \{ie{l,...,n}--Ui = u}\ 
n 

= for « e V s.t. i.^^;) > 0, (2) 

where we use the relation Vu\v{u\v)v^{v) — Vuv{u,v) even when v^{v) = 0. Let p and p' be probability 
distributions on the same set U and let q and q' be conditional probability distributions on the same set V. 
Then divergence D{p\\p') and conditional divergence D{q\\q'\p) are defined as 

p{u) 

■jya) lug 

D{q II q'\p) ^ E^^(^) E 9(^1") log 

For the proof of theorems, we use the method of type developed in [8], where we use the definition of a 
typical set introduced in [19] [27]. A set of typical sequences Tu,-y and a set of conditionally typical sequences 
7[/|y,^(v) are defined as 

Tu,-i = {u ■■ D{vu\\i^u) < 7} 
'Tu\v,'r{v) = {u : D{Pu\v\\lJ'U\vWv) < 7} , 
respectively. For u G Af", v G V", the entropy H{u), and the conditional entropy H{u\v) are defined as 

H{u\v) = J2''u\v{u\v)iyv{v)log- 

u,v '^u\v\"'\^) 

where Vu and v^^^ are defined as (1) and (2), respectively. For 7,7' > 0, we define 

_ |Z^|log(n + l) 



i«(7) = -\/2^1og^ (4) 
^i<|v(7'l7) = - v^log + V27log (5) 



^(,)._y2^1og^ + Mi^i^!i±ll (6) 
\U\ n 

vuiAi'h) - -^/2y log ^ + ^log \u\ + + (7) 

which comes from lemmas in Appendix B regarding the method of types. It should be noted here that the 
product set W X V is denoted by UV when it appears in the subscript of these functions. 
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For a mathematical statement S, we define x{S) as 



X{S) ^ { 



1, if S* is true 



0, if S is false. 



III. Formal Description of Problems and Known Results 



In this section, we review the problems of multiple access channel coding and results regarding achievable 
regions. 

A multiple access channel has k inputs and 1 output. Let /C be an index set of the channel inputs, where k = 
|/C|. Then the channel is characterized by the conditional probability distribution Hy\Xk^, where Xjc = {Xj}j^jc 
is a fc-tuple of random variables corresponding to the inputs and F is a random variable corresponding to the 
output. Let Xj be the alphabet of the j-th channel input and y be the alphabet of the channel output. 

In the following, we review some coding scenarios that will be discussed in subsequent sections. Let /C be 
an index set of messages and k = |/C|. For each i e /C, let A^j be the alphabet of the i-th message and Mi be 
the random variable corresponding to the i-th message, where we assume that the probability distribution of 
Mi is uniform on A4i for alH e /C. We also assume that random variables {Mi} .^^ are mutually independent. 
Let pMj^ be the uniform distribution on Adj^. We use the following notations: 

= {Vj}jeic 
Mj^= XMi 



Let cl( ) denote the closure of a region and co( ) denote the closure of the convex hull of a region. 
A. Private Messages 

In this scenario, we assume that JC = IC and there are k senders and k independent messages Mjc, where 
the j-th sender has access to the j-th message Mj and there is no common message. 

For a given block length n, a multiple access channel code {(pic, tp) (Fig.l) is defined by k encoders (pjc and 
one decoder -ip, where 




for given m, e Mi, i € 



/C 



(fj :Mj Xj- for each j € JC 



Then the error probability of the code is defined as 



Error((pK;,V') = I^Y\xAy\'P!c{mK))PMA'mic)x{i^{y) ¥= rriK:). 



The rate Rj of the j-th message is defined as 



log^ foreachieiC. 



September 21, 2012 



DRAFT 



6 



We call the rate vector Rjc achievable if for all 5 > and all sufficiently large n, there is a code ((^/c, V') with 
a rate vector Rfc such that 

Error ((^K.V') < ^• 

For a given {Mx, }jeK:> let 7^({/iXj }jeK) be the set of all fc-dimensional vectors i?/c satisfying 

< ^ < /(X^; y|X^c) for allj C /C, (8) 

where the joint distribution hx^y of random variable (Xjc, ^) is given by 

For given and {/^jc |c/}jeyc> let TZ{iJ,u, {l^x \u}jeJc) be the set of all /c-dimensional vectors Rjc satisfying 



0<^Rj < I{Xj- Y\U, Xjc) for allj c /C, 
where the joint distribution jiux^Y of random variable {U,Xiq,Y) is given by 



(10) 



HuxK.Y{u,XK:,y) = hyixAvI^ic) 



Huiu). 



(11) 



Y[ fiXj\u{Xj\u) 

Then the achievable region for this scenario is given as described below. 

Proposition 1 ([6, Theorem 15.3.6][10, Theorem 4.5]): The achievable region for this scenario is given as 



which is equivalent to 



CO I y n{{iix,}jGK.) I , 



[j cl {'R{iJ,u, {Mx, |c/}je/c)) , 



(12) 



(13) 



where \U\ <k. 

Remark 1: It should be noted that this proposition includes the result of |1||15] corresponding to the case 
of two encoders. The equivalence of the two regions (12) and (13) can be shown from [6, Theorem 15.3.6] 
and [10, Theorem 4.5] by considering the operational definition of capacity region. 

In Section V-A, for a given Rjc G TZ{^ij , {l^Xj\u}jGK), we construct a code with the rate vector R/c 
based on the coded time sharing technique. It should be noted that we can construct a code with a rate vector 
Ric e TZ{{^iXj}jeic) by letting J7 be a constant, that is, \U\ = 1. In fact, TZ{{^ix,}]eK:) = ^^{^J'U,{^^x,\u}3e^c) 
when [/ is a constant. The achievability of the region (12) with a proposed code can be proved by using the 
time- sharing argument. 

B. Multiple Common Messages 

In this scenario, we assume that there are k independent messages and k encoders, where the j-th 
encoder has access to the messages = {Mi}^^^ specified by JCj C JC for each j E IC. 

For a given block length n, a multiple access channel code {(picip) (Fig.2) is defined by k encoders (fjc = 
{>fij}jeJC and one decoder ^, where 



(fij -Mj^, for each j & K 
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Then the error probabiHty of the code is defined by 



For each i e /C, the rate Ri of the i-th message is defined by 

_ log|.Mi| 

Hi = . 

n 

We call the rate vector i?^ achievable if for all (5 > and all sufficiently large n, there is a code {(p)c, tp) with 
a rate vector i?^ such that 

Eiroi{(pic,tp) < S. 

For each i e /C, let be an auxiliary random variable and Xi is the alphabet of Xj. For a given {l^x }ieic 
and a set {fj}jeic of functions 

/j : Xj^^ — >■ Afj for each j G IC, 
let^ ^H({Mif .},£jc> {/jlje'c) ''^ s-dimensional vectors Rj^ satisfying 

0<^Ri< I{Xx; Y\Xx-) for all I C K, (14) 

where the joint distribution Hx_x^y random variable {Xj^,X)c, Y) is given by 



Then the achievable region for this scenario is given as described below. 

Proposition 2 ([12, Theorem 4.1]): The achievable region for this scenario is given as 

CO U ^H({MjfJjgjg,{/j}jeK;) 

where 



(15) 



< |/C| + JJ IA-jI for alii e/c. 



In Section V-B, for a given 'R-n{{tJ'x-^i&K.^{fi}i^K.)y we construct a code with the rate vector J?^. The 
achievability of region (15) with a proposed code can be proved by using the time-sharing argument. 

In the following, let us consider a scenario (Fig. 3) in which one of two senders has access to messages Mq 
and Ml and another sender has access to messages Mq and M2, where Mq denotes a common message. It is 
a special case of the above scenario, where K = {1, 2}, /C = {0, 1, 2}, /Ci = {0, 1}, and IC2 = {0, 2}. 

Let Ro be the encoding rate of the connmon message and Ri and R2 be the encoding rate of the private 
message of the respective encoders. Let'' TZswil^Xo^ fJ-XiiXo^ tJ-XzlXo) be the set of all {Rq, Ri, R2) satisfying 



Rq>0 



(16) 



'The subscript 'H' comes from the author Han of [12]. 

■^The subscript 'SW comes from the authors Slepian and Wolf of [26]. 
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0<Ri<I{Xv,Y\Xo,X2) (17) 

0<R2<I{Xr,Y\Xo,Xi) (18) 

Ri+R2<I{Xi,X2;Y\Xo) (19) 

R^ + Ri+R2< I{Xi ,X2]Y) (20) 

where the joint distribution UxoX^x^y of random variables (Xq, X\,X2-, Y) is given by 

tiXoXiX2Y{xo,xi,X2,y) = ^^Y\XlX2{y\x^^X2)^^Xl\Xo{xl\xo)nx2\Xo{x2\xo)nxo{xo). (21) 

It should be noted that (21) implies the fact that the right hand side of (20) is equal to I{XiX2Xq; Y). Then, 
the rate region is given as described below. 

Proposition 3 ([26]): For the scenario in which two receivers have access to their private message and a 
conmion message, the achievable region is given as 



CO 



[J TZ-swil^Xo, f^Xi\Xo, IJ'X2\Xo) j I (22) 

where 

|Afo| <min{|3;|+3,|Afi||Af2|+2}. 

Remark 2: It should be noted that region (22) is equivalent to the region obtained from (15). This has been 
proven in [12]. 

In Section V-C, for given {Rq, Ri, R2) G ^^sw{^J'Xa^ ^■l■Xl\Xo^ f-'-x-ilXo)' we construct a code with the rate 
vector (i?o, Ri,R2)- The construction is a typical example of the superposition coding introduced in [26] based 
on the hash property, and it is different from the construction presented in Section V-B. The achievability of 
region (22) with a proposed code can be proved by using the time-sharing argument. 

IV. (a, /9)-HASH PROPERTY 

In this section, we introduce the hash property first introduced in [22][21] and its implications. This notion 
is used for the proof of theorems. 

A. Formal Definition 

Here, we introduce the hash property for an ensemble of functions. It has been introduced in [22][21] and 
requires stronger conditions than those introduced in [19]. 

Definition 1 ([21] [22]): Lei A= be a sequence of sets such that A'^"'^ is a set of functions A : 

Im.4^"^ For a probability distribution pA,n on A^"\ we call a sequence {A,P/^) = {{A^"\ pA,n)}'^^=i 
an ensemble. Then, has a (qa, l3^)-hash property^ (or simply hash property) if there are two sequences 

aA = {aA(n)}^^i and ^a = {Mn)}'^^^, which depend on {pa,«}^=i, such that 

lim aA(n) = 1 (HI) 

n— >-cxD 

Um /3A(n) = (H2) 

n— >oo 

'in [21][22][23][24], it is called the 'strong hash property.' Throughout this paper, we call it simply the 'hash property.' 
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and 



Pf,,n {{A : Au = Av!}) < /3a (n) 



(H3) 



for any n and u e ZY". Throughout this paper, we omit the dependence of A, pa, ctA and /3a on n. 
Remark 3: In [19] [22], an ensemble is required to satisfy the condition 




where U is the range of functions. This condition is omitted because it is unnecessary for the results reported 
in this paper. 

Let us remark on the condition (H3). This condition requires the sum of the collision probabilities pa{{A: Au = Au'}), 
which is greater than aA/|Im>^|. to be bounded by 13a, where the sum is taken over all u' except u. 
For an ensemble of sparse matrices, aA represents the difference between {A.,Pa) and the ensemble of all 
linear matrices with uniform distribution, and 13a represents the upper bound of the probability that the set 
{u e : Au = 0}, which is called a code in the context of linear codes, has low weight vectors. It should 
be noted that this condition implies 



for any T, T' C W", which is introduced in [19]. The stronger condition (H3) is required for Lemmas 1 and 4, 
which will appear later. 

It should be noted that when A is a two-universal class of hash functions [9] and pA is the uniform distribution 
on A, then {A.,Pa) has a (1, 0)-hash property, where random bin coding [5] and the set of all linear functions 
[7] are examples of the two-universal class of hash functions. It is proved in [21, Section III-B] that an ensemble 
of sparse matrices has a hash property. From this fact, this ensemble of sparse matrices can be applied to all 
results in this paper. This impUes that all proposed codes can be constructed by using sparse matrices. 

We have the following lemma, where it is unnecessary to assume the linearity of functions assumed in 
[19] [20]. It is one of the advantages of introducing a stronger version of the hash property. 

Lemma 1 ([21, Lemma 4]): Let {A.,Pa) and {A.',Pa') be ensembles satisfying a (aA,/3A)-hash property 
and a (aA',/3A')"hash property, respectively. Let A G A. (resp. A' G A!) be a set of functions A : 1mA 
(resp. A' ■M'^ ^ ImA'). Let ^ = ^ x ^' and ^ = [A, A') e A defined as 



u'eT' 



PA {{A : Au = Au'}) < \TnT'\ + 



\r\\T'\aA 
\ImA\ 



+ min{|r|,|r'|}^A 



(H3') 



Au = (Au, A'u) 



for each A G A, u e W^. 




= aAQ!A' 



/3a =^A+^A'. 
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Fig. 4. Properties connecting the number of bins and items (black dots, messages), (a) Saturation property: every bin contains at least 
one item, (b) Collision-resistance property: every bin contains at most one item. 



B. Two Implications of Hash Property 

We review two implications of the hash property, which is introduced in [19]. These two implications connect 
the number of bins and messages (items) and are derived from the hash property by adjusting the number of 
bins taking account of the number of sequences. 

In the following, let ^ be a set of functions A : — > Im^, where an item is a member of U"'. A function 
A assigns a label Au of bin to an item u E U". 

Saturation property: We prepare a method that finds a typical sequence for each bin. The saturation property 
is a characteristic of the hash property. Figure 4 (a) represents the ideal situation of this property. When the 
number of bins is smaller than the number of black dots, we can find a suitable function whereby every bin has 
at least one black dot. This is because the hash property tends to avoid collisions. It should be noted that it is 
sufficient for coding problems to satisfy this property for 'almost all (close to probability one)' bins by letting 
the ratio [the number of bins] /[the number of black dots] be close to zero. To find a typical sequence from each 
bin, we use the minimum-divergence operation introduced in the construction of codes, where this operation 
finds a typical sequence when there is. In this situation, the black dots correspond to typical sequences. 
We have the following lemma, which is related to the saturation property. 

Lemma 2 ([19, Lemma 2]): Assume that the distribution of a random variable a is uniform on Im^ and a 
and A are mutually independent. If {A,p/k) satisfies (H3'), then 

PAa m, a) : r n CA{a) = 0}) < - 1 + l^^li^^ + M 

for any r C U"-. 

We prove the saturation property from Lemma 2. We have 

£a [Pc ({c : r n Ca(c) = 0})] = pAc {{{A, c) : T n Ca{c) = 0}) 

<.A-l+ ™ + ^] (23) 

By assuming that [Im>l|/|T| vanishes as n — >^ oo, we have the fact that there is a function A such that 

Pc{{c:TnCA{c)=(/)}) <6 

for any ^ > and sufficiently large n. Since the relation T fl C^(c) = corresponds to an event where there 
is no M e T in bin Ca{c), we have the fact that we can find a member of T in a randomly selected bin with 
probability close to one. 
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Collision-resistance property: A good code assigns a message to a codeword that is different from the 
codewords of other messages, where the error probability is as small as possible. The colUsion-resistance 
property is another characteristic of the hash property. Figure 4 (b) shows the ideal situation as regards this 
property, where the black dots represent messages we want to distinguish. When the number of bins is greater 
than the number of black dots, we can find a good function that allocates the black dots to the different 
bins. This is because the hash property tends to avoid the collision. It should be noted that it is sufficient for 
coding problems to satisfy this property for 'almost aU (close to probabiUty one)' black dots by letting the 
ratio [the number of black dots] /[the number of bins] be close to zero. This property is used to estimate the 
decoding error probability. In this situation, the black dots correspond to typical sequences. 

We have the following lemma, which is related to to the colUsion-resistance property. 

Lemma 3 ([19, Lemma 1]): If {A,pa) satisfies (H3'), then 

PA {{A : [g \ {u}] n Ca{Au) + 0}) < jg^ + /3a ■ 

for aU fir C W and u e W". 

We prove the colUsion-resistance property from Lemma 3. Let nu be the probability distribution on ^ c W". 
We have 

[l^u {{u : [g \ {u}] n Ca(Aw) ^ 0})] < J2 f^u{u)pA {{A : [fi? \ {u}] n Ca{Au) ^ 0}) 

ueg 

\g\cx/\ , . 



\ImA\ 



By assuming that |^|/|Ini>l| vanishes as n — )• oo, we have the fact that there is a function A such that 

flu {{u : [g \ {u}] n Ca{Au) 7^ 0}) < 5 

for any 6 > and sufficiently large n. Since the relation [g \ {u}] fl Ca{Au) ^ corresponds to an event 
where there is u' G g such that u and u' are different members of the same bin (they have the same codeword 
determined by ^4), we have the fact that the members of g are located in different bins (the members of g can 
be decoded correctly) with probabiUty close to one. 



C. Channel Coding Based on Hash Property 

Now, we explain an intuitive construction of a channel code in terms of the saturation property and the 
colUsion-resistance property, where the construction is introduced in [19]. 

We prepare two functions A : X"' — )• 1mA, B : Af" ImB, and a vector c e Im^, and assume that they 
are shared by an encoder and a decoder. It should be noted that |Im^| (resp. |ImB|) is the number of bins 
specified by A (resp. B). The function A is analogous to a parity check matrix in the context of linear codes. 
The set ImB is the set of aU messages and |ImB| is equal to the number of messages. 

The flow of vectors is is illustrated in Fig. 5. Let m e ImB be a message, x G Af" be a channel input, and 
y e iV" be a channel output. For c and a message m, a function "qab generates a typical sequence x G Tx,-y 
as a channel input, where Ax = c and Bx = m are satisfied. The decoder reproduces the channel input x by 
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Fig. 5. Construction of Channel Code 



using qa from c and a channel output y. Since {x, y) is jointly typical and Bx = m, the decoding succeeds if 
the amount of information of c is greater than H{X\Y) to satisfy the collision-resistance property. In fact, there 
are about 2"--^^-^^^^ conditional typical sequences x for given y and it is sufficient to prepare bins specified by 
A more than 2"^('^l^) to distinguish conditional typical sequences. Formally, this condition corresponds to 

'^^^ > H{x\n 

n 

On the other hand, the total rate of c and m should be less than H{X) to satisfy the saturating property. 

Since there is at most 2"^^'^) typical sequences, it is sufficient to prepare bins specified by A and B less than 

2ni?(x) Pormally, this condition corresponds to 

log|W||Img| ^ 
n < 

Since it is sufficient to satisfy these two inequalities, we have the fact that there is a code when 

^^^}^<H{X)-HiX\Y). 
n 

This implies that we can set the encoding rate of messages close to H{X) — H{X\Y) = I{X; Y). 
In this paper, we extend this approach to construct a multiple access channel code. 

D. Multiple Extension of Collision Resistance Property 

To prove the achievabiUty of a multiple access channel code based on the hash property, we extend the 
lemma related to the collision-resistance property. The hash property is needed to prove the following lemma, 
and this is another reason why the hash property is introduced. We use the following notations: 

CA^iajc) = {ujc : AjUj = aj for all j €lC}. 

AfcUK = {AjUj}.^^. 

For a C ZY" X V" and w e let and Qv\u{u) be defined as 

gu = {u: {u, v) eg for some v € V"} 
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Qv\u{u) = {v:{u,v)eg}. 
Furthermore, to shorten the description of the following lemma, we use the following abbreviation 

\g\, ifj = K 



\dJ\J'' 



max \Quj\Ujc{ujc)\ otherwise. 



(25) 



for Q c [Wk;]" and c /C. It should be noted that the expression \Qj\j<i \ does not represent the cardinality 
of the set Q j\j<:- 

Lemma 4 ([21, Lemma 7]): For each j G JC, let Aj be a set of functions Aj : ImAj and p/\^ be the 

probability distribution on Aj, where {Aj,pAj) satisfies (H3). We assume that random variables A^; = {^j}jeic 
are mutually independent. For each J d IC, let ocf^j and /J^j. be defined as 

Then 

for all Q C [Z^k]" and U}^ e [^jc]"- Furthermore, if {cxf^.,(3f^.) satisfies (HI) and (H2) for all j G JC, then 

lim aAj(n) = 1 (26) 

n— >-oo 

lim/3A^(n)=0 (27) 

n— fcxD 

for every J <zJC. 

V. Construction of Codes 
In this section, we construct codes for the scenarios introduced in Section III. 

A. Private Messages 

In this section, we consider a scenario in which k senders transmit independent messages to a receiver and 
there is no common message to be sent (Fig.l). 

First, we construct a code based on the coded time-sharing technique introduced in [13]. For a given ijly\Xk.'> 
Hu, and {iJLXj\u}3eK, assume that Rk. satisfies (10). Then there is {ejjjeyc such that 

E [Rj + Ej] < I{Xj-Y\U,Xjc) - e for allj c /C, (28) 

where e is defined as 



where 'nxK.\uy is defined by (7). For each j e K, let Vj be defined as 

rj^H{Xj\U)-Rj-ej. (30) 
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From (11), (28), and (30), we have 

Tj > I{Xj;Y\U, X,c\{j}) - Rj - Sj > 0. 

Let {A.j,p/i^.) and {A!j,p/i,i ) be ensembles of functions, and let Aj G A.j and A'j e A!j. Let Aj e Aj and 
A'j e Aj be functions 

Aj : ImAj 
4 : a;" ^Imyi; , 



respectively. We assume that ensembles satisfy 

log |Im^j 



n 



(31) 



log Im^'- 

R = _5J Jl. (32) 

n 

For each j e /C, let A^j be the set of messages defined as 

Mj = ImA'j. 

Then Rj represents the encoding rate of the j-th message. We assume that the j-th encoder and a decoder 
share functions Aj £ Aj, A'j e A'j and vectors Uj G ImAj and u € W". 
For each j e /C, we define the j-th encoder as 

fj ("^j ) = 9Aj A', {aj , rrij \ u) 
for a message ruj G Mj, where CAjA'.{o-j,mj) is defined as 

CAjA'.{'0>j,mj) = {xj : AjXj = aj and ^^ajj = rrij}. (33) 

and 

= arg min ^ -D(j^a;'>llMx,|c/kw)- 

We define the decoder as 

V'(y) = A'K9AAo'ic\y,u) 

for a channel output y Gy^, where 

dA^{aic\y,u) = aig min D{i^ua^>^y\\iJ,uxicY)- 

x'j&CAj (aj) 
for all je/C 

Figure 6 illustrates the code construction for k = 2. For given vectors Uj, u, and a message rrij, the function 
QAjA'. finds a conditionally typical sequence Xj satisfying AjXj = aj and A'jXj = rrij. The function Aj is 
analogous to the parity check matrix for the j-th message, and the function is a typical set decoder that 
guesses the channel input xk. satisfying AjXj = aj for all j e /C, where vectors a^, u, and a channel output 
y are given. 

Here, let us remark on the relations (28) and (30). From these relations and (11), we have 

rj + Rj = H{Xj I U) - Sj for all j e /C (34) 
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Fig. 6. Construction of Multiple Access Channel Code: Private Messages (Coded Time-sharing) 



and 

T.rj = ^[H{Xj\U)-R,-e,] 
jej jej 

= H{Xj\U)-Y,[R,+e,] 
jej 

> H{Xj\U) - I{Xj;Y\U,Xj.) + e 

= H{Xj\U,Xjc,Y) + e (35) 

for all J iz K,. Condition (34) is sufficient for the saturation property, that is, for a given u the j-th encoder 
can find a conditionally typical sequence corresponding to the j-th message rrij when the number 2"[''j+^j] 
of bins is smaller than the number of typical sequences. Condition (35) is sufficient for the collision-resistance 
property, that is, the decoding error probability goes to zero if the rate vector of the vector a^c is in the 
Slepian-Wolf region of the correlated source coding. It should be noted that the decoder can recover messages 
mic when the channel input xjc is successfully decoded by operating A'j^ to xk. because the j-th message rrij 
satisfies A'^Xj = rrij. 

For each j G /C, let Mj be a random variable corresponding to the j-th message, where the probability 
distribution pMj is uniform on Mj. Let Eiroi{Aic,A'i^,ajc) be the decoding error probabiUty. We have the 
following theorem. 

Theorem 1: Let MriX/c be the conditional probabiUty distribution of a stationary memoryless channel and 
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IJ-uxicY be defined by (11) for given probability distributions and {nxj\u}jeic- For given ii^: € TZ{fiu, {f^Xj\u} 
and {sj}jeic satisfying (31)-(29), assume that ensembles ) ("^iiPa') bave a hash property for 

all j e /C. Then, for any d > and all sufficientiy large n, there are functions (sparse matrices) {Aj}j^!c, 
{Aj}j^)C, and vectors {ajjjgK. w such that Aj e ^j, A'j e e Im^j, u e ZY", and 

Error( , ' ' < (36) 

Next, we construct a code with iix; € 7^({/iXj }jex;) by letting f/ be a constant, that is, \L{\ = 1. Although 
the result is straightforward, we describe the corollary which is used in the next section. Condition (10) is 
replaced by (8). Condition (28) is replaced by 

J2 [Rj + ^j] < y\Xjc) -e for allj c /C, (37) 



where e is defined as 



jeic 




(38) 



Definition (30) is replaced by 

rj = H{Xj) - Rj - Sj for all j S /C. (39) 
Functions QAjA'. and can be replaced by 

gAjA'iaj,mj) = &vg min L>(i/^J/xx,.) 

5AK;(aK;|y) = arg min D{v^'^y\nxKY), 

for all j'e/C 

respectively, where CAjA'.i<ij,'mj) is defined by (33). 

Figure 7 illustrates the code construction for k = 2. We have the following corollary. 

Corollary 2: Let /Uy|X/c be the conditional probabiUty distribution of a stationary memoryless channel and 
HxicY be defined by (9) for a given {nx^jjefc- For given Ric G 7^({/ix,}jeK;) and {ej}jeK, satisfying (31), 
(32), and (39)-(38), assume that ensembles {A.j,Pfi^.) and ) bave a hash property for all j e /C. Then, 

3 J j 

for any ^ > and all sufficiently large n, there are functions {Aj}j^ic> {Aj}jeic> and vectors {aj}j^!c such 
that Aj e Aj, A'j e Aj, Uj e ImAj, and Error ( A^;, AJ^-, a/c) < S, where Error(^A:>^A:'"A:) denotes the 
error probability. 

B. Multiple Common Messages 

In the following, we consider the scenario (Fig.2) where there are k messages and k senders transmit messages 
common to some users. 

In the following, we assume that for given fiY\XK^, {/"x liGJC' {fj}je!C, the rate vector Rj^ satisfies 
Rj^ e 7?.H({/Xj^ {fj}jeK.)- For a given fc-input multiple access channel ij,y\Xk:, let us consider a fc-input 
multiple access channel i^y\x-- defined as 
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Fig. 7. Construction of Multiple Access Channel Code: Private Messages 



Then the scenario of multiple common messages for the channel can be reduced to the scenario of 

private messages for the channel which the i-th input terminal has access to its private message Mi 

and there is no common message. Then, by applying Corollary 2 to the channel IJ-y\x~' i^SLve the fact that 
there is a code {(pj^,tp) for this channel at Rj^ satisfying (14). Figure 8 illustrates the construction of the code 
for the channel My| x ^ ■ ^ code {ifjc, ip) for the channel ij.y\Xk given as 

Hy) = i^iy) 

for a multiple message m^, where 

for each j and x^, = {xi}^^^.. Figure 9 illustrates the construction of the j'-th encoder, where we define 
% = l^jl- 

C. Two-user Multiple Access Channel Coding: Private and Common Messages 

In this section we consider a scenario (Fig. 3) in which one of two senders has access to messages Mq and 
Ml and another sender has access to messages Mq and M2. We construct a code based on a method that is 
analogous to a superposition coding. 
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Fig. 8. Reduction of Multiple Common Messages to Private Messages 



Encoder 
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Fig. 9. Construction of j-th Encoder 



For given Mr 1x1X2' Mxi|Xo' MxalXo' /"Xq. assume that (i?o, -Ri, -R2) satisfies (16)-(20) and the following 
three conditions 



Ro<I{Xo;Xi,X2,Y) 
Ro + Ri<I{Xo,Xi;X2,Y) 
Ro + R2<I{Xo,X2;XuY), 



(40) 
(41) 
(42) 



which will be eliminated by using the rate -splitting technique introduced later. Then there is e such that e > 
and 



Ro + so<I{Xo;Xi,X2,Y)-s 
Ri+ei<I{Xv,Y\Xo,X2)-e 
R2+si<I{X2;Y\Xo,Xi)-e 
Ro + Ri+eo+ei< I{Xo, Xi; X2,Y) - e 
Ro + R2 + £o + £2< I{Xo, X2;Xi,Y)-s 



(43) 
(44) 
(45) 
(46) 
(47) 
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Ri+R2 + si+S2<I{Xi,X2;Y\Xo)-e (48) 

i?o + ^1 + ^2 + £o + ei + £2 < IiXi,X2; Y) - s, (49) 

where e is defined by 



jeK. 



2^s,|. (50) 

jeic 



For each j G }C= {0, 1, 2}, let rj be defined as 

ro = H{Xo)-Ro-£o (51) 

ri=H{Xi\Xo)-Ri-ei (52) 

r2 = H{X2\Xo)-R2-e2. (53) 

From (21), (30), and (43)-(45), we have 

ro > /(Xo; Xi, X2, r) - i?o " eo > 
n >/(Xi;r|Xo,X2)-i?i-£i >0 
r2 > /(X2; Y\Xo, Xi) - i?2 - £2 > 0. 

For i G }C, let (^,^^4) (-^iiPA') be ensembles of functions, and let Ai G Ai and A'^ G A!^. Let G Ai 
and e A'i be functions 

A'^-.XP^ ImA',, 
respectively. We assume that ensembles satisfy 

log |Im^i| 



n 



(54) 



R, = (55) 

n 

Let A^i be the set of messages defined as 

Mi = ImA'i. 

Then (i?o, -Ri, -R2) represents the encoding rate of this code. We assume that, for each j G K. = {1,2}, the 
j-th encoder and a decoder share functions Aq g Aq, A'q g A'q, Aj G Aj, A'j G Aj, and vectors ao G ImAo 
and aj G Im^j. 

Let (mo, mi, 77x2) G -Mj^, be a multiple message. For each j G K, we define the j-th encoder as 

(Pj{mQ,mj) = gA^A'.{aj,mj\gAoA'g{ao,mo)), 

where 

9AoA'{aQ,mo) = arg min D{ua.'\\fiXo) 

gA,A'{aj,mj\xQ) = aig min D{iy^n^J\iix,\XoWxo), 
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Fig. 10. Construction of Multiple Access Channel Code: Private and Conunon Messages 



where CA^A'XO'j^'fnj) is defined by (33). We define the decoder as 

i^{y) = (^o.^'n^2)fl'AoAiA2(ao,ai,a2|y) 
for a channel output y Gy^, where 

9AoAiA2{ao,ai,a2\y) = a.ig min ^ D{iy^>^^'^^>^y\\iJxoXiX2Y) 

^o^C^o (ao) 
x[gCa:^ (ai) 
XjGCas {0-2) 

{A'q, A[, A'2){xo, Xi, X2) = {A'qXo,A[xi,A'2X2). 

Figure 10 illustrates the code construction. It should be noted that the construction is analogous to the super- 
position coding introduced in [4], where the function QaoA'^ finds a cloud center xo and the function Qa^a'^ 
finds a satellite Xj of the cloud center xq. 

Here, we remark on relations (30) and (43)-(49). From these conditions and (21), we have 

ro + Ro= H{Xo) - 60 (56) 
n+Ri=H{Xi\Xo)-ei (57) 
r2+R2=H{X2\Xo)-e2 (58) 
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and 

ro> H{Xo\Xi,X2,Y)+e (59) 

n> H{Xi\Xo,X2,Y)+s (60) 

r2> H{X2\Xo,XuY)+e (61) 

ro+n> HiXo,Xr\X2,Y)+e (62) 

ro + r2> H{Xo,X2\X,,Y)+s (63) 

n + r2 > H (Xi , X2 |Xo , y) + £ (64) 

ro+ri+r2 >i/(Xo,Xi,X2|F)+£. (65) 

Conditions (56)-(58) are sufficient for the saturation property, that is, 5aoA^ can find a typical sequence 
corresponding to the message mo, when the number 2"[''°+^ol of bins is smaller than the number of typical 
sequences. Similarly, gAiA' can find a conditionally typical sequence for a given xq corresponding to the i-th 
message mj when the number 2"['"*+^'l of bins is smaller than the number of conditionally typical sequences. 
Conditions (59)-(65) are sufficient for the colhsion-resistance property, that is, the decoding error probability 
goes to zero if the rate r)c of the vector aic is in the Slepian-Wolf region of the correlated source coding. It 
should be noted that when the channel input (xq, xi, X2) is successfully decoded the decoder can recover the 
i-th message m, by operating A'^ to Xi because rrii satisfies A^Xi = rrii. 

For each i G )C, let Mj be a random variable corresponding to the i-th message, where the probability 
distribution is uniform on A4i. Let Error(^£, a^) be the error probabiUty of this code. We have the 
following theorem. 

Theorem 3: Let fJ-y \X1X2 be the conditional probabihty distribution of a stationary memoryless channel 
and fJ.XoXiX2Y be defined by (21) for given probability distributions /iXo> Mxi|Xo' I^X2\Xo- For given 
(ro,ri,r2), {Ro,Ri,R2), and {eo,£i,E2) satisfying (30) and (43)-(50), assume that ensembles {^j,Pfi^.) and 
{A.'^,p/^i ) have a hash property for all j e /C. Then, for any 5 > and all sufficientiy large n, there are 
functions {Aj}^^j^, {Aj}j^j^ and vectors {aj}j^j^ such that Aj e Aj, A'j G A'j, aj G Im^j, and 

Error , A~ , a^) < S. (66) 

In the following, we employ a rate splitting technique to eliminate conditions (40)-(42). Assume that 
{Ro,Ri,R2) satisfies conditions (16)-(20) and (40)-(42). From Theorem 3, we have the fact that there is 
a code with encoding rate {Ro,Ri,R2)- Let mo € be a common message and mi e Af"^^ and 

m2 e X2^^ be the private messages of two different encoders. We divide the private messages into two parts 

/ n[Ri-R'/] nR''\ 

mi ~ [mi ,rni I 

/ n[R2-R'2] nR'i\ 

m2 = [nil ,m2 1 , 



where {R'{,R^) satisfies 

< R'l < Ri (67) 

< E2 < i?2. (68) 
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Let us interpret ^mo,mj ^ ,7723 ^ j as the common message and rrij 
j-th encoder. Then we have the fact that rate {R'q, R[,R'2) satisfying 



nlRi-R';] 



as the private message of the 



Rq — Ro + R'l + R-z 



(69) 



(70) 



R'2 — R2 — R'i 



(71) 



is achievable by using the same code obtained from Theorem 3. 

Now we prove the fact that for all {R!^, R'l^R'^) G T^swilJ'Xo, IJ'Xi\Xoi lJ'X2\Xo) there is a pair {R'(, R2) such 
that {Ro,Ri,R2) satisfies conditions (16)-(20), (40)-(42), (67)-(71). From (69)-(71), we have 



By substituting these inequalities into (16)-(20), (40)-(42), (67), and (68), we have 

-^0 — -^1' — R2 — ^ 
0<R[+ R'l < /(Xi; y 1X0, X2) 
< R2 + R2 < I{X2;Y\Xo,Xi) 
R[ + R'i + R'2 + R!i < I{Xi,X2; Y\Xo) 
R'o + R[ + R^<I{Xo,Xi,X2;Y) 
R'o ~ R'l ~ R'i < -^(-'^0; X\^X2, Y) 
R'o + -^'1 ^ ^2 < -^(-''^0, Xi;X2,Y) 
Ro+ R'2~ R'i < -'^(-^Oj X2;Xi, Y) 
< R'l < R'l + R'l 
^ R'2 ^ R'2 ~\~ R^i 

where we use the relation I{Xq, Xi,X2; Y) ~ I{Xi,X2; Y) obtained from (21) in the fifth inequality. By elim- 
inating R'l and R'2 from these inequalities by using the Fourier-Motzkin method (see [10, Appendix D][28]) and 
the relation /(Xi; X2IX0) — obtained from (21), we have the fact that {R'q, R'^, R'2) G 7^svk(mxoi A*:^i|Xo j A^XslXo)- 
This implies that for {R'^, R'^, R'2) G 'R-swipxa, ^iXi\XoJ f^X2\Xo)y there is (i?'/,i?2) such that (i?o, -Ri, 
defined by (72)-(74) satisfies (i?o, i?2) G nsw{^^Xo, ^^x^\Xo, f^^x^lXo)^ (40)-(42), (67), and (68). This 
means that we can construct codes with {R'q,R'i,R'2) G T^-sh^C/^jCqi A*^i|Xo' MXal^o)- Thus, conditions (40)- 
(42) are eliminated. 

VI. Proof of Theorems 

In this section, we prove Theorems 1 and 3. Before describing the proof, we remark on the outUne of 
the proof. The proof is similar to the conventional random coding argument, where a codebook is randomly 




(72) 



R\ — R'^ -\- Ri 



(73) 



R2 — R2 ~\~ R2' 



(74) 
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generated and it is proved that the average error probability tends to zero as n goes to infinity. However, there is 
a definite difference from the conventional random coding argument in the following proof because our proof is 
based on random partitioning and the probability distribution of a codebook is different. This will be explained 
in detail later. 

In the following proof, we omit the dependence of X, Y,U on n when they appear in the subscript of fi. 

For u = {ui,...,Un) and (x/c,Z/) = {{{xijjjetcVi), ■ ■ ■ ,{{xnj}jeic,yn)) e [Xk]" x 3^", l^u{u) and 
IJ-YiXiciylxK:) are defined as 



n 

IJ-YlX^ivlxic) = '[ll^Y\xAy^\{^^J}3elc)■ 



A. Proof of Theorem 1 

In the following, we assume that ensembles {A.j , Pa ) and (A' , p^' ) have a hash property for all j G /C. 
Then, from Lemma 1, ensemble defined by 

has a (a^ ,/3^ )-hash property, where 
Since 



lim ^A;c(^) — 

n—>oo n— >-oo 

= 0, 



n[/3A,+i]-i 



there is a sequence k = {K(n)}^i such that 

lim K(n) = 00 

n—yoo 

lim /3A^(n) =0 

n—>oo 

lim = 0, 

where k is the number of encoders. For example, we obtain such a k by letting 

n^/'' if 3^ > s.t. /3ak; {n) = o {n-^/'') 

[/^A^cC")]"^^'*"^^^ otherwise 
for every n. If /3ak(^) is not o (n"''/*^), there is a k' such that k' > 0, /3ak: (^)^^^*' > and 

logK(n) _ log^i^ 



K(n) = < 



(75) 

(76) 
(77) 

(78) 



n 



[k + 1] 
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log 

~ [k + l]n 

£, log n log k' 



(79) 



k[k + l]n [k + l]n 

for all sufficiently large n. This implies that k satisfies (78). In the following, k denotes K{n). 
From (78), we have the fact that there is a 7 such that 7 > and 

[fc + 3]7 + E^^.I"(^W^E^J- (81) 
jeic jefc 

for all j E IC and sufficiently large n. 
When u S Tj/,^, we have 

= K|Im^j||Im^j-| 

> «;|Im^j| (82) 

for all j e /C and sufficiently large n, where the first inequality comes from Lemma 13, the second inequality 
comes from (80), the first equality comes from (31) and (32), and the last inequaUty comes from the fact that 
Im^j C Im^j X ImA'j. 
This implies that for all j e /C and sufficiently large n there is Tj{u) C Txj\u,'y{u) such that 

^ < < 2n. (83) 

|Im^j| 

for all u. We assume that Tj{u) is constructed by selecting |7j(ti)| elements in the ascending order regarding 
the value -D(t'a,,.|„||Mx,|c/ku). 

Let mic € Adic be private messages. Let xjc be channel inputs, where Xj G Af" is defined as 

Xj = gAjA'. {C'j^'mj \u) for each j E IC. 

Let y e be a channel output. We define 

Sj = {{mK.,y) : Xj e Tj{u) and y e 3^"} 

Sk+i = < {mK,y) : I{xk\u) < 7 + Vxi\u{l\l) + ^3] and y e 

Sk+2 = {(m/c,y) : y e TY\uxK,iiV'^^ic)} 
Sk+3 = {{m!c,y) : gAK:{a!c\y,u} = xk:} , 

where j &1C and 

I{xk\u) = ^H{xj\u) - H{xjc\u). 
jeic 
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We assign equation numbers to the conditions 

Xj eTj{u) cTx^\u,i{u) forallje/C 

I{x!c\u) < 7 + X] [''xM^h) + £j] 
ie/c 

y e Ty\ux^,'({u,xk.) 

9AK:{aic\y,u) ^ XK, 



(84) 
(85) 
(86) 

(87) 
(88) 



which are referred later. Since the j'-th message rrij satisfies A'^Xj = rrij, the decoder can recover message 
m/c when decoding the channel input Xfc is successful. This implies that the decoding error probabihty is 
upper bounded by 

ErTor{Ajc,A'^,aic,u) 

< E pmM^j) + PM^Y ( [nU^j] n + PMM^k+2) + vm„y ( [n^^^^,] n 5^+3) . (89) 

Remark 4: The condition (86) was unnecessary in the proof of the conventional random coding argument 
because a;^; € ^x^\u was naturally satisfied by generating codewords independently at random for a 
given u E U^. On the other hand, (86) is necessary in our proof because (84) and (85) may not imply 
{u,xic) € TuXfz.-y' for an appropriate 7' > 0. This is the difference from the conventional proof. It should be 
noted that (84)-(87) implies {u,Xfc,y) G TuXfzV.j', where 7' > will be specified later. 

In the following we evaluate the average error probability 



< E- 



A/ca/c 



E Hu (w)Error(A/c , , a^; , u) 



< 



A/ca>c 



jetc 



+ E- 



E ^iu{u)pM^Y{S]) 
E tJ-u{u)pM^Y{S^+2) 



+ E: 



+ E- 



AjcaK 



E fiu{u)pM^Y {[n^=iSj] n 

E nu{u)pM.Y{[n^t'^Sj]nS^,+,) 

U&Tu,-y 



(90) 



over random variables A^;, a/c, and ?7". The last term on the right hand side of (90) is evaluated as 

^lu {[TunD < 2-"[T-^"l 



< 



(91) 



for all 5 > and aU sufficiently large n, where the first inequahty comes from Lemma 12. In the following, let 

Sj = {a j, rrij) for each j G IC. 

We assume that the distribution of Sj is uniform on ImAj for all j e /C, and random variables {Aj, a^, Mj}j^jc 
and f/" are mutually independent. In the following, we use the fact that 5^ a'. (a^, mjlu) ^ Tj{u) implies 
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Tj{u)r\C^, (Sj) = 0, which is shown by contradiction as follows. Let us assume that Xj = QAjAr {(^jjfnj\u) ^ 
Tj{u) and Tj{u) r\C^,{aj) ^ 0. Then there is x'j G Tj{u) fl C^(aj). From the definition of "gAjA'., we have 

Dii'^^luWl^x.iuW-a) < D{l^^>^\^\\lIXj\uWu)- (92) 
On the other hand, from the construction of Tj{u), we have the fact that x'J e Tj{u) if Xj e Tj{u) and 

D{iya:'J\u\\HXj\uW-u) < D{u^', 

\u\\MXj\uWu)- 

From this fact and (92), we have the fact that Xj E Tj (u), which contradicts the assumption Xj = qAjA'. , mj\u) ^ 
Tjiu). First, we evaluate E^^^^ J2ueTu.~, f^u{u)pM^Y{S;j) 



^ IJuiu)pM^Y{S]) 



< 



From Lemma 2 and (83), we have 
l^u{u)p^^^^j^^ ({(Aj,4,aj,mj) : gA,Ar{aj,mj\u) (j, 7;(u)}) 

E '"C^(")PA,a, ({ (^i. ^i) ■ Tji^) n ^A, (Si) = 0}) 



ueTu,' 

< ' 



llmAi 



"a, - 1 



"a, - 1 



/3a.. + 1 



/3a,. + 1 



(93) 



for all (5 > and sufficiently large n, where the first inequality comes from the fact that gAjA'.{cij,mj\u) ^ 
Tj{u) impUes Tj{u) r\C^,{aj) = 0, and the last inequality comes from (76) and the fact that a-^, — )• 1 and 
B-s — ^ as n — >■ 00. 

Next, we evaluate the second term on the right hand side of (90). Assume that xic satisfies (84), (85), and 

I{x!c\u) > 7 + X] [''Xjiuilh) + £j] • 
jeic 

Then, from Lemma 11, we have 

\H{xj\u) - H{Xj\U)\ < i;^^\u{l\l) for all j e /C. 



We have 



H{xic\u) = X H{xj\u) - I{xk:\u) 



<Y,[HiXj\U) + L;,^\u{7h)] 



jeic 



where the last equality comes from (30). Since xj e ^AjA'. (%> ''^i) for all j G /C, we have 

XK€g{u)nC^^{aic), 



(94) 
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where G{u) C ^-jeic -^f is defined as 



This implies that 



Then we have 



g{u) = \ xk: : H{x!c\u) < ^[rj + i?^] - 7 ^ . 

jetc 



P^KBK {{i^ic,aK) : g{u)nC^^{aj^) ¥=<!>})< Yl P^^S^ ({(1^,2^:) : Ajxj = a/or all j e /c}) 



= E n 

= E 



< 



_ 2-n[7-A;v;cl, 

where the second inequahty comes from Lenmia 8. This impUes that 



(95) 



E- 



Y i^uiu)pM^Y{[n';^iSj]n 



s 



< 



k + 4 



(96) 



for all (5 > and sufficiently large n, where the last inequality comes from the fact that Xx^: — > as n — > 00. 

Next, we evaluate the third term on the right hand side of (90). Let X/c = {gAjA'. {^jjMj\u)}jeK- Then we 
have 



fJ-YlXK: {[TY\UXK:,-f{u,X!c)Y \Xk.) = tJ.Y\U''XK {[TY\UXK.,ii'^-, ^ K.)]" \u, X k) 



(97) 



from Lemma 12. This implies that 



E- 



= E^ 



Y fJ'Uiu)lJ,Y\XK, {[Ty\UXk.,7{^^^>c)Y\Xk.) 



5 



< 



k + 4: 



(98) 
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for all 5 > and sufficiently large n, where the last inequality comes from the fact that Xux^y as n oo. 

Next, we evaluate the fourth term on the right hand side of (90). In the following, we assume that (84)-(87) 
and 

9Aic{0'K:\y,u) ^ Xic.X 

Then, from (11), we have 

+ > v^{u) log i-T + > v-u.^^ (m, xk) log — ■ r~rT 

< [fc + 2]7 + 7 + ^ [ix,\u{l\l) + ej] 
jeK. 

<2^ej (99) 
je/c 

where the last inequality comes from (81). This implies that 

{u,Xfc,y) e Tx^Y,Y> 

where 7' is defined as 

jeic 

Since gA^(aic\y,u) ^ xk., there is x'^ G CA^{ajc) such that x'^^ ^ xk. and {u,x'^,y) G T[/Xx;F,7'. This 
implies that 

[g{u,y)\{XK}]f^eAAAKXK) 

where 

= {a;/c : {u,XK.,y) e 7[/XK;r,7'}- 

From Lemma 9, we have the fact that 

Q{u,y) c Tx,c\UY,y{u,y) 
and xx: € Q{u,y) imphes {u,y) G Tuy,j'- Then, from Lermna 13, we have 

\gK\Ko{u,y)\ = \g{u,y)\ 

< \Tx^\uY,yiu,y)\ 

^ 2n[Ii{^K.\UY)+Tixi^luyh'h')] _ (100) 

For each non-empty set J C.K, let 

Gxjc {u, y) = {xjc : Xk: € ^^(m, y) for some xj- e X'^} 
Gxj\Xjo{u,xja,y) = {xj : Xk: G ^(m,^)}. 
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Then, from Lemma 9, we have the fact that xjc & Gxjc {u, y) imphes {u, xjc,y) g TuxjcY,j' and 

Gxj\Xjc{u,xj^,y) c Txj\ux:rcY,Y {u,xjc,y) 
for every non-empty set J C.K. We have 

\Qj\J''{'^^v)\ = _ _ _^ \gxj\Xjc {u,xjc,y)\ 

\Txj\uXjcY,^' {u,xjc,y)\ 



< max 

(u,a!jc,i/)eruxj.cy,7' 



(101) 



for every non-empty set J where the second inequality comes from Lenoma 13. Then, from (100), (101), 
and Lemma 4, we have 



E/K^ [x{9^A^KXK\y.u) ^ x^)] < {{Aic ■ [Q{u,y) \ {xK.}]r\CAAAKXK) ^ 0}) 



< 



E 

JdK. 

for all (ti, Xfc,y) € TuxkY,i'- Then we have 



2n[HiXj\UXjc,Y)+r>x^,^y(^'W)\^^^ [/3a^c + l] 

rijejrllmAI 



+ (102) 



< f^u{u)tJ.Y\xAy\^ic) 



EkK 



jeic 



Y m{u)lJ.Y\xAy\XK)EA^ [x(5Ax;(AKa;/c|y,w) 7^ Xk)] 



yeTY\x^.-,(^K.) 



< 



1 
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JCK 



< 



fc + 4 



(103) 



for all (5 > and all sufficiently large n, where Tk.{u) is defined as 

jeic 

the equality comes from Lemma 5 that appears in Appendix A, the third inequality comes from (102), the 
fourth inequality comes from (31) and (83), and the last inequality comes from (26), (27), (35), and (77). 

Finally, from (90)-(93), (96), (98), and (103), we have the fact that for all 5 > and all sufficiently large n 
there are {Aj, A'^^a^jj^/c, and u satisfying Aj e A, A'^ e A'^, aj € ImAj, u gU"' and (36). ■ 

B. Proof of Theorem 3 

We can prove the theorem similarly to the proof of Theorem 1. 

In the following, we assume that ensembles {A.j , pp^. ) and {A.'j , pp^i, ) have a hash property for all j € /C. 
Similarly to the proof of Theorem 1, we define an ensemble {A.j,p'^ ) and (a^ ,/3^ ) for each j e /C. Then 
we have the fact that {A.j,p^) has a (a^ ,/3^ )-hash property and there is a sequence k = {^(n)}^^! such 
that 



lim K(n) = 00 

n— >oo 

lim [K{n)fp^Jn) = Q 



lim 

n— >oo 



log K{n) 



From (106), we have the fact that there is a 7 such that 7 > and 

, , log K 



, . . log K, 



for all j e /C and all sufficiently large n and 



(104) 
(105) 
(106) 

(107) 
(108) 

(109) 



where tj (7) is defined by 



Similarly to the proof of (83), from (107), we have the fact that there is a set To such that % C Txo,-y and 



K < 



< 2k. 



(110) 



\lmAo\ 

We assume that To is constructed by selecting |To| elements in the ascending order regarding the value 
D{i'xq\\ij.Xo)- Furthermore, from (108), we have the fact that for all xq G and all j e /C there is a 
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set Tj{xo) such that Tj{xo) C Tx^\Xo,'y{xo) and 

«<rZH^<2K. (Ill) 
|Ini>li| 

We assume that Tj{xo) is constructed by selecting |7^(a;o)| elements in the ascending order regarding the value 

-DK,>ollMx,|Xoka=o)- 

Now we prove the theorem. Let mo € Ado be a connmon message and mi G A4i and m2 € be private 
messages. Let xq € be defined as 

a;o = gAoA'g{ao,Tno)- 
Let (xi,a;2) be channel inputs, where Xj G Af" is defined by 

Xj = gAj A'. {aj,mj\xQ) for each j G IC. 
Let y e 3^" be the channel output. We define 

So = {{mj^,y) : ajo e To C Txo,f and y Gy"} 

= {("T'/c'y) • S 7j(a=o) C 73i:^|Xo,7('^o) and y G 3^"} 

«53 = < {m^,y) : /(xi;x2|xo) < 7 + X! W and y e ^'^ 

^4 = {(m£,y) : y e TVix^.^Ca^jc)} 
^5 = {(m^,y) : 5Ag(a^|y) = Xj^} , 

where j gK, and 

7(xi;a;2|a;o) = ^ if(a;j|a;o) - i?(a;{i,2}|xo) 
je{i,2} 

= H{xi\xo) + if(a;2|a;o) - H{xi,X2\xo). (112) 
The error probabihty is upper bounded by 

Error(^^,A^,a^) 

jeic 

(113) 

We assign equation numbers to the conditions 

xoGToCTxon (114) 
Xj G Tjixo) C Tx,\xo,j{xo) for all i e /C = 1, 2 (115) 

/(a;i;a;2|a;o) < 7 + X! I'^J^^) + ^^'J ^^^^^ 
jeic 

yGTY\x^,^{xj^) (117) 

gA^{aj^\y) ^ xj^ (118) 
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which are referred later. In comparison with the conventional superposition coding, the condition (114) corre- 
sponds to an event where the function QaoA'^ finds a 'good' cloud center xq and the condition (115) corresponds 
to an event where the function QajA'^ finds a 'good' satellite Xj for all j G K, where 'good' means that they are 
(conditionally) typical sequences. When (114)-(117) are satisfied, we have the fact that (a;o,a;i,a;2) is jointly 
typical. It should be noted that (116) was unnecessary in the proof of the conventional superposition coding 
because the joint typicality of {xq, xi,X2) was naturally satisfied by generating codewords at random. 
In the following, let 

Sj = {aj,mj) for each j &K. 

We assume that the distribution of cLj is uniform on Imylj for all j G /C, and {A^, a^, Mj}^.g£ are mutually 
independent. 

First, we evaluate E^^g_ [pMi^YiSg)] . From Lemma 2 and (110), we have 

-^A^ag [PMi^riS^)] =PAoaoMo ({(^0, ^0, mo) : 9AoA'„{ao,mo) i To}) 
<PAoao({(^o,So):ronC^^(ao) = 0}) 

I Wo I ko + i 



<"ao-1 + 



ITol 



^ "Ao - 1 



< 



/^Ao + l 



(119) 



for all ^ > and sufficiently large n, where the last inequality comes from (104) and the fact that a^^ 1 
and Bti — )• as n — )• oo. 

Ao 

Next, we evaluate a [pm^y{So n<S|)]. From Lennma 2 and (111), we have 

^^A^aJfM^V-(5on<S|)] 

= ^0^0(^0,00) X (?l„(ao) = xo) PA,a, ({ ■■ ?l,(«jko) i Tj{xo)Yj 

H PAoao(^o,ao) ^ (saS^o) = xo) p^.^. ({(A,-,a,) : Tjixo) nC^,{aj) = 0}) 

|W,| +1 



< 



^Oiao 



< 



H PAoao(^o,ao) X (?Ao("o) = a^o) 



Aoiao 



< a. . - 1 



< 



. - 1 + 



\rj{xo)\ 



(120) 



for all (5 > and sufficiently large n, where the last inequaUty comes from (104) and the fact that — 1 
and B-T — )• as n ^ 00. 

Next, we evaluate E-^^ [pm^y {[n'j^QSj] fliSg)]. Assume that {xo,xi,X2) satisfies (114), (115) and 

I{xi ; a;2 |a;o) > 7 + ^ [ij (7) + Sj] . 
jeic 
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Then, from Lemma 11, we have 



Then we have 



\H{xo)-H{Xo)\<iXoh) 
\H{x^\xo) - H{X^\Xo)\ < i^:,iXo{lh) 
\H{x2\xo) - HiX2\Xo)\ < t;t.|A-o(7l7)- 

H{xo,xi,X2) = H{xo) + H{xi\xo) + H{x2\xo) - I{xi;x2\xo) 



< HiXo) + H{Xi\Xo) + H{X2\Xo) 



7 + ^[ij(7) + ei] 



= Eh+'Rj]-7, 

where the last equality comes from (51)-(53). Since Xj G CajB^ {O'j^fnj) for all 3 S /C, we have 



(121) 



{xo,xi,X2) G 5nC^_(a£), 



where Q (Z x x X^ is defined as 

\ {xo,xi,X2) ■■ H{xo,x-i,X2) < ^[rj + Rj] -7 
This impUes that 

Similarly to the proof of (96), we have 

- H ^'Agag • ^i^i = «J for all i e X;}) 

|g| 

2" 



< 



2-n[7-A^-] 



- 6 



(122) 



for all (5 > and all sufficiently large n. 
Next, we evaluate \pm^y(SI)\. Similarly to the proof of (98), we have 



Ty\Xc.,j{X^) 



(123) 
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for all ^ > and all sufficiently large n, where = {Xj} .^j^ is defined by 

X j = gAj A'j (a j , Mj- 1 Xo ) for each j & IC. 
Next, we evaluate -E^^-a- [P'^k'^ ([^1=0*^^] ^"^s)]- ^ following, we assume (114)-(117) and 

gAoAiA2{ao,ai,a2\y) ^ {xo,xi,X2). 
Similarly to the proof of (99), we have 

D{u^-y II Hx^y) = D{Vy\^^ II llY\x^\yxg) + XI -^('^»=.>o II tojIXoka^o) +^Ko II UXo) + I{xi]X2\xq) 

je{o,i} 

<57 + 5^[i,-(7)+e,-] 

3&K 

<2^e„ (124) 
where the last inequality comes from (109). This implies that 

where 7' is defined as 

jelc 

Since 5^0^1^2(00,01,0211/) ^ (a;o,a;i,a;2), there is {x'q,x[,x'2) e CAg(ajg) such that {x'q,x[,x'2) ^ 
{xo,xi,X2) and {x'Q,x[,x'2,y) G Txj^Y,^- This implies that 

[0{y)\{xjc}]nCA^{Aj^xj^)j^9, 

where 

g{y) = {(a;o,a:;i,a;2) : {xo,xi,X2,y) e Txj^Y,j'} ■ 
From Lermna 9, we have the fact that 

Q{y) c Txj^\Y,j'{y) 

and (xo, ici, 0:2) e t/(y) impUes y e Ty,^'- Then, from Lemma 13, we have 

ie£|£c(y)| = |e(y)| 

< |73f^|y,7'(y)l 

^ 2"[^^(Xg|F)+r,;,-|y(7'|7')]_ Q25) 

For each non-empty set J C.JC, let 

Qxjc (y) = {a;j-c : x^ G Q{y) for some xjGXj} 
Qxj\Xj.{xjc,y) = [xj : x^ e g{y)] . 
Then, from Lemma 9, we have the fact that xj<^ € Qxjo (y) implies {x j<=,y) € TxjcY,'^' and 

^'A'j|A'j=(a;j'c,y) C 73fj|Xj=F,7' {xjc,y) 
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for every non-empty set J C.K,. We have 



\Qj\J''{v)\ = max \gxj\x^a{xj.= ,y)\ 



max \Txj\XjcY,'^'{xj''.y)\ 

^ 2n[H{Xj \Xja ,y) + ,,;v^ I y (7' It')] 

^ 2n[H{Xj\Xjo,Y)+nx-\y(l'W)] 



(126) 



for every non-empty set J C.1C, where the second inequality comes from Lemma 13. Then, from (125), (126), 
and Lemma 4, we have 

E^^ [x{g^^{^jcx^\v) + < {{Aj^ : [g{y) \ {x;^}] nC^^(A^x^) 0}) 



^ 2n[mx,ix,.,Y)+n.,^yiVlV)]^^^ ^^^^^ ^^j 



jac 



(127) 



for all {xj^, y) e Txj^Y,-y'- Then we have 



XI x(5Ao(ao) =a;o) 



X '^i'l^g (y l^£)x(5A^ (a£ |y) 7^ Xjc) 



3/eTy|Xg,7(a!K) 



je/c 



< X MF|X;e(y|a=ic)'^A^ 

l/err|Xg,7(a!)c) 

" n -II A I ^ ^iY\x^{y\x^)E^^ [x{g^^{^^^x^\y) + x^)\ 



< 



X f^Y\x^{y\xf^] 



1' Xj^£T 

l/eTy|X^,7(a!g) 



Ff TFZTT-i + PAf 



< 



E2 

JCK 



(128) 



for all 5 > and all sufficiently large n, where T is defined as 

T= {(£Co,a;i,a;2) : xq &To,xi £ Ti{xo),X2 e 7^(a;o)} , 

the equality comes from Lemma 5, which appears in Appendix A, the third inequality comes from (127), the 
fourth inequality comes from (54), (110), and (111), and the last inequality comes from (26), (27), (59)-(65), 
and (105). 

Finally, from (113)-(120), (122), (123), and (128), we have the fact that for all ^ > and sufficiently large 
n there are {Aj,Aj,aj}j^^ satisfying Aj e A, Aj e A'j, aj G Im^j, and (66). ■ 
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Appendix 

A. Basic Property of Ensemble 

Lemma 5 ([19, Lemma 9]): Assume that random variables A and a are independent and the distribution of 
a is uniform on Im^. Then, 

= = ^ 

for any A £ A and u e W", and 

E.3[x(An = a)] = ^ 

for any u G U". 

B. Method of Types 

Let Tu CU" he a set of all sequences that has the same type uu, where type of m e U"' is defined by the 
empirical distribution Let Tij^j be a set of typical sequences and 7[/|y,^(t^) be a set of conditionally typical 
sequences defined in the begirming of Section II. 

Lemma 6 ([8, Lemma 2.2]): The number of different types of sequences in ZY" is fewer than [n+ IJI^L The 
number of conditional types of sequences in U"' x V" is fewer than [n + IJI^H^I. 

Lemma 7 ([8, Lemma 2.3 and 2.5]): Let Xu be defined in (3). Then 

2nlH{U)-Xu] <\ru\< 2""'-^K 

Lemma 8: For H >0, 

I {u : H{u) <H}\< 2"[^+^"l 

where Xu is defined by (3). 

Proof: The proof is similar to that of [20, Lemma 6]. We have 

\{u:Hiu)<H}\= J2 I'^^l 

U:H(U)<H 

< ^ 2nH{U) 
U:H(U)<H 

U:H(U)<H 

< [n+l]l"l2"-f^ 

= 2"[^+'^"l, (129) 

where the sum is taken over all random variables U corresponding the type of a sequence in W^, the first 
inequality comes from Lemma 7, and the last inequality comes from Lemma 6. ■ 

Lemma 9 ([19, Lemma 22][27, Theorem 2.5]): If t; S 7V,7 and u G Tu\v,y{''^)' thsn {u,v) G Tuv,j+j'- If 
{u,v) e Tuv,-y, then u e Tu.y and u e 7c/|i/.- (w). 

Lemma 10 ([27, Theorem 2.6]): If m G 7c/,7> then 

ueii 
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Proof: The statement is shown by 



ueu 



< 



27 



log2e 
<^/27, 

where e is the base of the natural logarithm and the first inequality comes from [6, Lemma 12.6.1]. 
Lemma 11: Let < 7 < 1/8. If v G 7V,7> and u G 7[/|y,-y'(v), then 

\H{v) - H{V)\ < Lvil) 
\H{u\v) - H{U\V)\ < iuwil'bl 

where lu and t;^|y are defined by (4) and (5), respectively. 
Proof: From [8, Lemma 2.7], we have 

\H{p)-H{q)\<-e\og-^^ 

for any 9 and probability distributions p and q onV satisfying 

Y.\v{v)-q{v)\<e<\. 
vev 

Then the first inequality is shown by this fact and Lemma 10. 

Next we prove the second inequality. Let yu\v^v and ijlu\vI'v be defined as 

yu\vV.v{u,v) = Vu\.v{u\v)iy^{v) 
IJ.u\v'^v{u,v) = iiu\v{u\v)i'v{v), 



respectively. Since 



Vu\v{u\v) 

liu\v{u\v) 



= D{v^\^ II lJ,u\v\Vv) 



we have 



\H{v^\^\v.o) - H{ij,u\v\vv)\ 

= E -u\Au\v)Mv) log ^^-i^ - E mvi^\v)Mv) log -^^-i^ 



v{u\v) 



= ^i'u\v{u\v)v^(v)\og r^'i''? , . -Y\lJ'U\v{u\v)v^{v)\og ^''\'"] 

^ v^\^{u\v)v^{v) ^ iJ.u\v{u\v)u^{v) 
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where the last inequality comes from (131). We have 



\H{iyu\vWv) - H{iy^i^\iJ,v)\ 



< E W^iv) - fiv{v)\ E ''■u\v{u\v) log - 

V 

< v^loglZ^I, 



(134) 

where the last inequality comes from Lenmia 10 and the fact that H{i'u\v{'\v)) < log|ZY|. From (133) and 
(134), we have 



\Hiu\v) - H{U\V)\ < - H{^iu\vK)\ + \H{f,u\vW.) - H{U\V)\ 

< i-uivii'b)- 



(135) 



Lemma 12 ([19, Lemma 26][27, Theorem 2.8]): For any 7 > 0, and v G V", 

fiu\v{[Tuiv,j{v)r\v)<2-^^''-^"^\ 

where Xu and Xuv are defined in (3). 
Lemma 13 ([19, Lemma 27] [27, Theorem 2.9]): For any 7 > 0, 7' > 0, and v G 7v,j, 



-log\Tu,-y\-H{U) 



n 



-\og\Tu\vn'{'")\-H{U\V) 



where r/w(7) and ?fe/|v(7'|7) are defined in (6) and (7), respectively. 
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